In
mathematics, an information source is a sequence of
random variables ranging over a
finite alphabet Γ, having a
stationary distribution Stationary distribution may refer to:
* A special distribution for a Markov chain such that if the chain starts with its stationary distribution, the marginal distribution of all states at any time will always be the stationary distribution. Assum ...
.
The uncertainty, or
entropy rate
In the mathematical theory of probability, the entropy rate or source information rate of a stochastic process is, informally, the time density of the average information in a stochastic process. For stochastic processes with a countable index, the ...
, of an information source is defined as
:
where
:
is the sequence of random variables defining the information source, and
:
is the conditional
information entropy
In information theory, the entropy of a random variable is the average level of "information", "surprise", or "uncertainty" inherent to the variable's possible outcomes. Given a discrete random variable X, which takes values in the alphabet \ ...
of the sequence of random variables. Equivalently, one has
:
See also
*
Markov information source
In mathematics, a Markov information source, or simply, a Markov source, is an information source whose underlying dynamics are given by a stationary finite Markov chain.
Formal definition
An information source is a sequence of random variables ...
*
Asymptotic equipartition property
In information theory, the asymptotic equipartition property (AEP) is a general property of the output samples of a stochastic source. It is fundamental to the concept of typical set used in theories of data compression.
Roughly speaking, the th ...
References
* Robert B. Ash, ''Information Theory'', (1965) Dover Publications.
zh-yue:資訊源
Information theory
Stochastic processes
{{statistics-stub